Weighted sigmoid gate unit for an activation function of deep neural network
نویسندگان
چکیده
منابع مشابه
Deep Gate Recurrent Neural Network
This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit (DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and le...
متن کاملTwo-steps Implementation of Sigmoid Function for Artificial Neural Network in Field Programmable Gate Array
The complex equation of sigmoid function is one of the most difficult problems encountered for implementing the artificial neural network (ANN) into a field programmable gate array (FPGA). To overcome this problem, the combination of second order nonlinear function (SONF) and the differential lookup table (dLUT) has been proposed in this paper. By using this two-steps approach, the output accur...
متن کاملSigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
In recent years, neural networks have enjoyed a renaissance as function approximators in reinforcement learning. Two decades after Tesauro's TD-Gammon achieved near top-level human performance in backgammon, the deep reinforcement learning algorithm DQN achieved human-level performance in many Atari 2600 games. The purpose of this study is twofold. First, we propose two activation functions for...
متن کاملVerification of an Evolutionary-based Wavelet Neural Network Model for Nonlinear Function Approximation
Nonlinear function approximation is one of the most important tasks in system analysis and identification. Several models have been presented to achieve an accurate approximation on nonlinear mathematics functions. However, the majority of the models are specific to certain problems and systems. In this paper, an evolutionary-based wavelet neural network model is proposed for structure definiti...
متن کاملGauss-sigmoid neural network
Recently RBF(Radial Basis Function)-based networks have been widely used because they can learn a strong non-linear function faster and more easily by their local learning characteristics. However, it has no hidden units that can represent some global information. Accordingly even if the knowledge obtained through the previous sets of learning is utilized effectively in the present learning, th...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Pattern Recognition Letters
سال: 2020
ISSN: 0167-8655
DOI: 10.1016/j.patrec.2020.05.017